Papers with zero-shot and fine-tuned settings
Paraphrase Identification via Textual Inference (2024.starsem-1)
Copied to clipboard
| Challenge: | Paraphrase identification (PI) and natural language inference (NLI) are important tasks in natural language processing. |
| Approach: | They propose a method for paraphrase identification and natural language inference using an NLI system to solve these tasks. |
| Outcome: | The proposed method outperforms dedicated PI models on PI datasets and provides insights into limitations of current benchmarks. |
SenseRel: A Sense-Level Benchmark for Denotational and Connotational Meaning Relations (2026.acl-long)
Copied to clipboard
Pierluigi Cassotti, Naomi Baes, Stefano De Pascale, Jáder Martins Camboim de Sá, Francesco Periti, Nick Haslam, Dirk Geeraerts, Nina Tahmasebi
| Challenge: | Polysemy enables a single word to convey multiple related meanings . a word's sense is extended to new contexts and concepts, a process called semantic change is gradual . |
| Approach: | They propose a benchmark for modeling semantic relations between word senses . they use a model that distinguishes denotational and connotationally related aspects of meaning . |
| Outcome: | The proposed model is able to distinguish between denotational and connotationalist aspects of meaning . it is compared with models with GPT-4o, Llama 3.1, and DeepSeek . |
Musical Score Understanding Benchmark: Evaluating Large Language Models’ Comprehension of Complete Musical Scores (2026.acl-long)
Copied to clipboard
Congren Dai, Yue Yang, Krinos Li, Huichi Zhou, Shijie Liang, Zhang Bo, Enyang Liu, Ge Jin, Hongran An, Haosen Zhang, Peiyuan Jing, KinHei Lee, Zhenxuan Zhang, Xiaobing Li, Maosong Sun
| Challenge: | Existing benchmarks for musical score understanding are narrow in scope, focusing on isolated fragments, short excerpts, or multiple-choice formulations, rather than supporting holistic reasoning over entire scores. |
| Approach: | They propose a benchmark for score-level musical understanding across textual and visual modalities. |
| Outcome: | The musical score understanding benchmark contains 1,800 question-answer pairs from works by Bach, Beethoven, Chopin, Debussy, and others. |
J-Shuwa: A Large-Scale Web-Collected Japanese Sign Language-Japanese Parallel Corpus (2026.findings-acl)
Copied to clipboard
| Challenge: | Japanese Sign Language (JSL) is a low-resource sign language that has received limited attention in the AI community due to the lack of large-scale, publicly available parallel corpora. |
| Approach: | They propose a large-scale JSL-Japanese parallel corpus constructed from YouTube videos with hard-coded subtitles and closed captions. |
| Outcome: | The proposed model is effective for training models and can be used for future research across a wide range of tasks. |